GPTR: Gestalt-Perception Transformer for Diagram Object Detection

Diagram object detection is the key basis of practical applications such as textbook question answering. Because diagram mainly consists simple lines and color blocks, its visual features are sparser than those natural images. In addition, diagrams usually express diverse knowledge, in which there many low-frequency categories diagrams. These lead to fact that traditional data-driven model not suitable for this work, we propose a gestalt-perception transformer detection, based on an encoder-decoder architecture. Gestalt perception contains series laws explain human perception, system tends perceive patches image similar, close or connected without abrupt directional changes perceptual whole object. Inspired by these thoughts, build graph encoder, composed nodes relationships between edges. This aims group into objects via similarity, proximity, smoothness implied edges, so meaningful can be effectively detected. The experimental results demonstrate proposed GPTR achieves best task. Our also obtains comparable over competitors detection.